The human eye offers a fascinating window into an individual's personality traits, medical problems, brain abnormalities, behavioral conditions, cognitive attention, and decision making. These characteristics have made it the subject of decades of research by experts in cognition, ophthalmology, neuroscience, epidemiology, behavior, and psychiatry, who have not only enhanced the understanding of how the eye works, but also revealed new ways of diagnosing health concerns. For example, nearly every health condition that affects the brain causes substantial variations in eye movement patterns including attention deficit hyperactivity disorder (ADHD), Autism, Williams syndrome, Schizophrenia, Parkinson's, Alzheimer's disease, Depression, and others.
Despite the enormous potential for advancing detection of health states and understanding of human decision making by measuring the eye, progress has been stymied by the lack of wearable eye trackers that are integrated into a regular pair of eyeglasses. The design of a low-power wearable eye tracker is remarkably challenging from the computation, sensing, communication, and aesthetic design perspectives. A real-time eye tracker involves an eye facing imager sampling at frame rates of tens of Hz (up to 100 Hz to detect fine-grained eye movements or saccades) thereby generating megabits of data per second and making communication to a phone extremely power-hungry. As a reference point, the Google Glass optical head-mounted display lasts only a few hours when streaming from its outward facing camera, while running too hot for comfort. Real-time computation on the eyeglass is also remarkably challenging, particularly given the volume of data and complexity of the image processing techniques. While the focus in is on the computation and power aspects, aesthetic design presents an equally significant challenge since the sensors need to be embedded in an unobtrusive manner within an eyeglass frame.
While many eye trackers are available for commercial use, they are expensive, power hungry and relatively bulky, which makes them less than ideal for regular use. Several efforts have been made to design low-power wearable eye trackers, but many challenges remain. For example, power consumption is a major avenue for improvement in eye trackers with some consuming 1.5 W and more optimized eye trackers consuming around 70 mW. These numbers are still much higher than typical wearables which only consume a few milliwatts of power, so there is a need to enable long-term operation of eye trackers on small wearable batteries.
Another desired improvement is robustness. Eye trackers simply do not work outdoors given the variability in outdoor lighting conditions. More generally, achieving robust operation in environments with different illumination conditions is extraordinarily challenging and has not been achieved so far by either research prototypes or bulkier commercial products. While some eye trackers rely on visible light, they fail under poorly illuminated conditions. Many commercial eye trackers use near-infrared (NIR) illumination of the eye, but fail to operate effectively outdoors due to overwhelming ambient infrared light.
Continuous real-time tracking of the state of the eye (e.g. gaze direction, eye movements) in conjunction with the field of view of a user is profoundly important to understanding how humans perceive and interact with the physical world. Real-time tracking of the eye is valuable in a variety of scenarios where rapid actuation or intervention is essential, including enabling new “hands-free” ways of interacting with computers or displays (e.g. gaming), detection of unsafe behaviors such as lack of attention on the road while driving, and leveraging visual context as a signal of user intent for context-aware advertising. Continuous eye tracking is also useful in non-real-time applications including market research to determine how customers interact with product and advertising placement in stores, and personal health, where the state of the eye provides a continuous window into Parkinson's disease progression, psychiatric disorders, head injuries and concussions, and others.
While wearable devices provide insight into a variety of physiological and health conditions, one aspect that has lagged behind is the ability to infer an individual's cognitive state in real-time. There is significant need for a device that can continuously monitor fatigue, since this has implications for a wide range of safety-critical application domains. For example, in 2014 there were 846 drowsy-driving-related fatalities (2.6% of all fatalities) recorded in the National Highway Traffic Safety Administration's (NHTSA) Fatality Analysis Reporting System (FARS) database. Being alert and conscious is not only crucial for drivers, but also for many other sensitive and important tasks. For example, if air traffic control (ATC) operators become distracted or their performance impaired, thousands of people could be put in danger. Similarly, other safety critical jobs include heavy machinery operators, power plant operators, and so on. Relying on self-awareness is not sufficient according to experts, both drivers and employees in sensitive positions fail to recognize their fatigue and drowsiness state. Therefore, there is a tremendous need for real-time, precise detection of the fatigued state in order to eliminate hazardous situations. Eye monitoring is a promising technology to address the aforementioned problems.
Eye movements are impacted directly by the underlying neural mechanisms of the brain, and therefore can provide useful information about the cognitive state of individuals. In order to continuously monitor the eye, remotely mounted cameras (e.g. on the dashboard of a car) may be used or eye measurement sensors embedded in a pair of spectacles. While the more common technology today is remotely mounted dashboard cameras for monitoring fatigue and drowsiness, these units rely on a user being in front of the camera.
Eye monitoring can be a much more viable technology for monitoring fatigue and drowsiness if the eye monitoring capability can be embedded in a regular pair of spectacles. Thus head-mounted eye monitoring is a much more viable technology for monitoring fatigue and drowsiness. Advantageously, wearable devices can continuously monitor eye movement in mobile settings useful for a broad range of applications such as detecting fatigue among medical professionals, detecting fatigue among shift workers, and others.
In addition, continuous measurements can also provide more information throughout the day, thereby providing advance warning about fatigue state prior to performing a safety critical activity such as driving. Furthermore, monitoring the eye in close proximity makes such a method more robust to the issues that are faced by remote eye monitors such as occlusions, head movements, variability in ambient lighting, and reflections due to spectacles or sunglasses. Thus, head mounted monitors can potentially provide higher accuracy detection than remote monitors.
Accordingly, there is a need for an eye tracking system that reduces power consumption and displays a more robust usefulness, such as for use in outdoors conditions. The present disclosure addresses these needs.